Automated Software Conformance Testing
نویسنده
چکیده
English) Software has found its way into our everyday lives and as a consequence the impact of software failures can range from just being inconvenient to being life-threatening in the worst case. Consequently, software quality is a major concern in the development of modern software systems. Testing is considered as a practical technique for finding bugs in computer programs. An important aspect to be considered is conformance testing, which is used to validate the correctness of an implementation with respect to its model. In such a setting a conformance relation formalizes the notion of correctness. One commonly used conformance relation is the input-output conformance (ioco) relation. This thesis aims to provide theories and techniques that facilitate the application of ioco testing in an industrial context. Originally, the input-output conformance relation was formulated over labeled transition systems and so far some of the underlying assumptions have only been stated on an informal basis. This thesis reflects the input-output conformance relation to reactive processes of the Unifying Theories of Programming framework. Thereby, all assumptions of ioco are formalized. Furthermore, this thesis shows how some of these assumptions can be relaxed. Knowing the pros and cons of a particular testing technique leads to the next step in conformance testing: the selection of an appropriate set of test cases. One promising test case selection technique in the context of ioco testing is the use of formal test objectives, i.e. test purposes. However, nowadays such test purposes are usually developed by hand which makes the quality of the resulting test suite dependent on the skills of the tester. This thesis investigates techniques that support test engineers in complementing manually generated test purposes and by automatically deriving test purposes. Once a set of test cases has been generated and executed a lot of time is spent on the analysis of the test execution results. This thesis describes an approach that intends to assist test engineers in grouping test cases with respect to detected failures. By analyzing only one test case per group the time and effort needed for the test result analysis can be significantly reduced. To sum up, this thesis makes input-output conformance testing more amenable to practical implementation.
منابع مشابه
Exploiting Timed Automata for Conformance Testing of Power Measurements
For software development, testing is still the primary choice for investigating the correctness of a system. Automated testing is of utmost importance to support continuous integration and regression tests on actual hardware. For embedded systems, power consumption is a chief performance metric, which is tightly coupled to the hardware used and the software exploiting low power modes. Automated...
متن کاملAutomatic Conformance Testing of Internet Applications
We adapt and extend the theories used in the general framework of automated software testing in such a way that they become suitable for black-box conformance testing of thin client Internet applications. That is, we automatically test whether a running Internet application conforms to its formal specification. The actual implementation of the application is not taken into account, only its ext...
متن کاملA Formal TLS Handshake Model in LNT
Testing of network services represents one of the biggest challenges in cyber security. Because new vulnerabilities are detected on a regular basis, more research is needed. These faults have their roots in the software development cycle or because of intrinsic leaks in the system specification. Conformance testing checks whether a system behaves according to its specification. Here model-based...
متن کاملRandom Visual GUI Testing: Proof of Concept
Market demands for higher quality software and shorter time-to-market delivery have resulted in a need for new automated software testing techniques. Most automated testing techniques are designed for regression testing that limit their fault finding ability to faults explicitly tested in scenarios/scripts. To overcome this limitation, companies define test processes with several test technique...
متن کاملA Two-Level Approach to Automated Conformance Testing of VHDL Designs
For manufacturers of consumer electronics, conformance testing of embedded software is a vital issue. To improve performance, parts of this software are implemented in hardware, often designed in the Hardware Description Language VHDL. Conformance testing is a time consuming and error-prone process. Thus automating (parts of) this process is essential. There are many tools for test generation a...
متن کامل